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Abstract 
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In this paper, we propose new sequential estimation methods based on inclusion principle. The main 
1 idea is to reformulate the estimation problems as constructing sequential random intervals and use con- 

fidence sequences to control the associated coverage probabilities. In contrast to existing asymptotic 
r, ' sequential methods, our estimation procedures rigorously guarantee the pre-specified levels of confi- 

| dence. 

1 Introduction 

i " , ' An important issue of parameter estimation is the determination of sample sizes. However, the appropriate 

sample size usually depends on the parameters to be estimated from the sampling process. To overcome 
this difficulty, an adaptive approach, referred to as sequential estimation method, has been proposed in 
sequential analysis, where the sample size is not fixed in advance. Instead, data is evaluated as it is 
collected and further sampling is stopped in accordance with a pre-defined stopping rule as significant 
. results are observed. In the area of sequential estimation, a wide variety of sampling schemes have been 

t— I " proposed to achieve prescribed levels of accuracy and confidence for the estimation results. Unfortunately, 

, existing sequential estimation methods are dominantly of asymptotic nature. That is, the guarantee of 

the pre-specified confidence level comes only as the margin of error approaches zero or equivalently the 
£N| ■ average sample size tends to infinity. Since any practical sampling scheme must employ a finite sample 

size, the application of asymptotic sequential methods inevitably introduce unknown statistical error. 
To overcome the limitations of existing asymptotic sequential estimation methods, we shall develop new 
sampling schemes by virtue of the inclusion principle proposed in [51 [7J . 

In our paper [5J [7J , we have demonstrated that a wide variety of sequential estimation problems can 



in 



- 1—1 

X 



5^ 

" be cast into the general framework of constructing a sequential random interval of a prescribed level of 

coverage probability. To ensure the requirement of coverage probability, we propose to use a sequence 
of confidence intervals, referred to as controlling confidence sequence, to define a stopping rule such that 
the sequential random interval must include the controlling confidence sequence at the termination of 
the sampling process. In situations that no other requirement imposed on the sequential random interval 
except the specification of coverage probability, we have proposed a more specific version of this principle for 
constructing sampling schemes as follows: The sampling process is continued until the controlling confidence 
sequence is included by the sequential random interval at some stage. Such a general method of constructing 
sequential estimation procedures is referred to as Inclusion Principle, which can be justified by the following 
probabilistic results. 

Theorem 1 Let (Q, J^", {^e}, Pr) be a filtered space. Let r be a proper stopping time with support I T . For 
£ £ I T , let »4.£ and Be be random intervals defined by random variables measurable in J?~£. Assume that 
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{t = £} C {.A< C S £ } /or £ G I T . Then, Pr{9 £ B T } > Pr{6> £ A e for I £ I T } > 1 - ^ £g/t Pr{6> £ A} /or 
any real number 9. 

See [5] for a proof. This theorem implies that the coverage probability of the sequential random 
interval constructed based on the inclusion principle is bounded from below by the coverage probability of 
the controlling confidence sequence. 

The remainder of the present paper is organized as follows. In Section 2, we shall apply the inclusion 
principle to develop analytic stopping rules for estimating the parameters of binomial, geometric and 
Poisson distributions. For wider applications, we address the problem of estimating a binomial proportion 
in a more general setting of estimating the mean of a bounded random variable. To make the stopping 
rules as simple as possible, we have made effort to eliminate the need of computing confidence limits. In 
Section 3, we further consider the problem of estimating the mean of a bounded random variable by taking 
into account the information of sample variance. Section 4 is the conclusion. The justification of stopping 
rules and proofs of theorems are given in Appendices. The main results of this paper have appeared in our 
conference paper 

Throughout this paper, we shall use the following notations. Let "A V B" denote the maximum of A 
and B. Let N denote the set of positive integers. Let R denote the set of real numbers. Let Pr{i?} denote 
the probability of event E. The expectation of a random variable is denoted by E[.]. The other notations 
will be made clear as we proceed. 



2 Analytic Stopping Rules 

In this section, we shall propose various analytic stopping rules for estimating mean values of random 
variables with pre-specified precision and confidence levels. More formally, let X be a random variable 
with mean E[X]. The general problem is to estimate ¥,[X] based on i.i.d. samples Xi,X2, ■ • • of X by 
virtue of sequential sampling. For n £ N, let X n denote the sample mean of X, i.e., X n = ^'tJ ' ■ When 
the sampling process is terminated with sample number n, the sample mean X a is taken as an estimate 
for E[X]. To describe our stopping rules, we need to introduce some bivariate functions as follows. 
Define function ^#b(- 7 •) such that 



^b(z, 9) 



Define function j#g{., .) such that 




z) In ±5f forz e (0,1), 9 £ (0,1), 
for z = 0, 9 £ (0,1), 
for z = 1, 9 £ (0,1), 
for z £ (-oo, oo), 9 (0, 1). 




{l-z) In iff for z e (1, oo), 6 £ (1, oo), 
Jt G {z,6) = { -ln<9 for z = 1, 9 £ (l,oo), 

for z £ [1, oo), 9 £ (1, oo). 



Define function ^#p(., .) such that 



'z-6» + zln(f) for 2 > 0, 6 > 0, 
Jt v {z, 9)= {-9 for z = 0, 9 > 0, 

oo for z > 0, 9 < 0. 



2.1 Estimation of Means of Bounded Random Variables 

Let X be a random variable such that K[X] = /x £ (0, 1) and < X < 1. Let S £ (0, 1). To estimate /x 
with a margin of absolute error e £ (0, ^) and confidence level 1 — 8, we consider sampling procedures of s 



2 



In — 

stages. Let nil, m 2 , • • ■ , m s be an ascending sequence of positive integers such that m s > 2 | . Let JV be 
a subset of positive integers which contains {mi, ■ ■ • , m s }. We propose two stopping rules as follows: 

Stopping Rule A: Continue sampling until ./#b (j^ ~ \ \ — X n | + e — „" T £ „ f , \ — \ \ — X n | + s^j < In ^ 

for some integers n £ JY and £ £ {1, • • • , s}. 

l 2 



X n -k -e 



2 I 1 3(nVm,) 



4 ^nVm( J 21 



> \ - -rr— for some 



Stopping Rule B: Continue sampling until 

integers n £ jV and £ £ {1, • ■ • , s}. 

In Appendices IA.1I and IA.21 we have shown that for both stopping rules A and B, the sample mean 
X n at the termination of the sampling process guarantees that Pr{|AT n — /i| < e} > 1 — S. To avoid 

In 2s 

unnecessary checking of the stopping conditions, we suggest choosing mi > : — $— for Stopping Rule A and 

m i > ^ 24g ~ 16e ^ for Stopping Rule B, respectively. For purpose of efficiency, we recommend choosing 

mi, • • ■ , m s as a geometric sequence, i.e., "l^ 1 is approximately equal for I = 1, ■ • • , s — 1. 

Next, consider the problem of estimating fi with a margin of relative error e G (0,1) and confidence 
level 1 — 8. Let 81,62, - • • be a sequence of positive numbers such that X)fc=i 81 = 8 E (0, 1). Let mi , m,2, ■ ■ • 
be an ascending sequence of positive integers such that lim£_ i . 00 in ^^ = 0. Let jV be a subset of positive 
integers which contains {mi,m,2, ■ • • }. We propose a stopping rule as follows: 

Stopping Rule C: Continue sampling until X n > and (l + n \/m t ) > T+i) — ~m~t m ^ ^ or some 

integers n£ / and I £ N. 

In Appendix IA.31 we have established that for stopping rule C, the sampling process will eventually 
stop with probability 1 and the sample mean X n at the termination of the sampling process guarantees 
that Pr{|X n - fj,\ < efj,} > 1 - 8. 



2.2 Estimation of Means of Geometric Distributions 

Let X be a random variable having a geometric distribution with mean 9 £ (1, 00). Let e, 8 £ (0, 1). To 
estimate 9, we consider sampling procedures of s stages. Let mi,rri2, ■ • ■ ,m s be an ascending sequence 
of positive integers. Let JV be a subset of positive integers which contains {mi,-- - ,m s }. Under the 
assumption that m s > 1 wc P r °posc the following stopping rule: 

Stopping Rule D: Continue sampling until f fl + £ — n ym l X n , (1 + s)X n ^j < ^ In ^ for some 

n £ JY and I £ {1, ■ • • ,s}. 

In Appendix IA.41 we have proved that for stopping rule D, the sample mean X n at the termination 
of the sampling process guarantees that Pr{(l — e)X n < 9 < (1 + e)X„} > 1 — 8. To avoid unnecessary 

In — 

checking of the stopping condition, we suggest choosing mi > j^n^fi) for Stopping Rule D. For purpose 
of efficiency, we recommend choosing mi,-- - ,m s as a geometric sequence. It should be noted that the 
estimation of a binomial proportion p with a margin of relative error e can be accomplished by such method 

if - is identified as 9. 

p 

2.3 Estimation of Poisson Parameters 

Let X be a random variable having a Poisson distribution with mean A £ (0, 00). Let e > and < 8 < 1. 
To estimate A, we consider sampling procedures of infinitely many stages. Let 81,82, - ■ • be a sequence of 
positive numbers such that Yl^-i 8j = 8 £ (0,1). Let mi,m,2,--- be an ascending sequence of positive 
integers such that lim^oo - = 0. Let ,jV be a subset of positive integers which contains {mi, m-2, ■ • • }. 

To estimate A with a margin of absolute error e and confidence level 1 — 8, wc propose the following 
stopping rule: 

Stopping Rule E: Continue sampling until t -#p (^X n + e — n ^ n( , X n + e^j < ^ In 4f for some integers 
n£ jV and t £ N. 

In Appendix IA.5[ we have established that for stopping rule E, the sampling process will eventually 
stop with probability 1 and the sample mean X n at the termination of the sampling process guarantees 
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that Pr{|A n — A| < e} > 1 — <5. To estimate A with a margin of relative error e and confidence level 1 — 5, 
we propose the following stopping rule: 

Stopping Rule F: Continue sampling until X n > and ^y+J (l + n vfn~ e ) ' — ml m if ^ or some 

integers n e and IgM. 

In Appendix IA.61 we have established that for stopping rule F, the sampling process will eventually 
stop with probability 1 and the sample mean X n at the termination of the sampling process guarantees 
that Pr{|A n - A| < s\} > 1 - 6. 



3 Estimation of Means and Variances of Bounded Variables 

In Section T2.ll we have proposed sequential methods for estimating the mean of a bounded random variable. 
However, the information of sample variance is not used in these methods. In this section, we shall 
exploit the information of sample variance for purpose of improving the efficiency of estimation. To 
apply the inclusion principle to construct an estimation procedure for estimating the mean of a bounded 
random variable, we need to have a confidence sequence for the mean. The construction of the required 
confidence sequence can be accomplished by applying Bonferroni's inequality to a sequence of fixed-sample- 
size confidence intervals. Therefore, in the sequel, we shall first study the construction of fixed-sample-size 
confidence intervals for the mean and variance of a bounded random variable. 

Since any bounded random variable can be expressed as a linear function of a random variable bounded 
in [0, 1], it will loss no generality to consider a random variable X bounded in interval [0, 1], which has 
mean /i g (0, 1) and variance a 2 > 0. Let X\, ■ ■ ■ ,X n be i.i.d. samples of X. Define X = x ' and 

y _ 2^ i= i\ t j n marL y situations, it is desirable to construct confidence intervals for /i and a 2 based 

on X and V. For this purpose, we need to make use of Hoeffding's inequalities. Specifically, define 

I a\ (^ ZV \ 6 + v(v-Z) zv z 

(£(z, v, ) = 1 In H In — 

^ V ' ' 1 V v + 6 J 6 v 2 + 6 v 

for < z < v < 1 and < 9 < 1. Define tp(z, v, 9) = 92(1 - z, 1 — v, 8) for < v < z < 1 and < 9 < 1. 
Define <f)(z, 0) = (1 — z) In + z In | for < z < 1 and < 6 < 1. Hoeffding's inequalities assert that 

Pr{A > z} < cxp(~nip(z, n,a 2 )) < exp(— n(f>(z, /i)) for < fj, < z, 
Pr{X < z} < cxp (— rnp(z, fi, er 2 )) < exp(— n(j)(z, fj,)) for z < fx < 1. 

We have the following results. 

Theorem 2 



dip{z,fi,9) 

d[i 
dili(z, fi, 9) 

09 

d<p{z,n,9) 

d<p(z,fj,,8) 
89 



< for0<fj,<z, (1) 

< for < 6 < 1, (2) 
> for < z < n, (3) 

< for < 6 < 1. (4) 



See Appendix [B] for a proof. 



3.1 Confidence Interval for Mean Value 

For simplicity of notations, define W v = V + (X — v) 2 for < v < 1. For constructing a confidence interval 
for the mean, we have the following method. 
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Theorem 3 Let 6 £ (0,1). Define 



U = 



supj^ e (0,X) : max [ip(X, v, ti), cf>{W v , ■&) l{ 4> w v }] > ^ /or aZ/tf G (0, u(l - v)]\ ifX>0, 

ifX = 

inf {z/ G (X, 1) : max [v9(X,i^,i9), 0(^,0) I{^>W„}] > ^ /or alM € (0, - i/)]} i/X < 1, 

1 i/X = 1, 



where ^{s>w„} is ^ e indicator junction which takes value 1 if § > W u and otherwise tales value 0. Then, 
Pr{L < fx < U} > 1 - 5. 

See Appendix [C] for a proof. The computation of the confidence limits is addressed in the sequel. 
3.1.1 Adaptive Scanning Algorithms 

To compute the lower confidence limit L, we first need to establish a method to check, for a given interval 
[a, 6] C [0, X], whether the following statement is true: 

l n 3 

For every v G [a, 61, max \ib(X, (j>(W„,$) Ii# >w x] > — - for all i? e (0, v(l - iA] . (5) 

n 

To check the truth of ((5]) without exhaustive computation, our approach is to find a sufficient condition 
for ([5]) so that the conservativeness of the sufficient condition diminishes as the width of the interval [a, b] 
decreases. For simplicity of notations, let c = max{a(l — a), 6(1 — 6)}. As a consequence of ([T]), we have 
4>{X,v, •&) > ip(X,b, for all v G [a, 6]. Since Wb < W v < W a for v G [a, 6] and </>(z,$) is non-increasing 
with respect to z G (0,$), we have (f>(W u ,'d)l^ >Wv y > <f)(W ai r d)l^ >Wa y for v G [a, 6]. Hence, a sufficient 
condition for (O is as follows: 

l n 3 

max [i/)(X,b,0), 4>(W a ,ti) I { # >Wa }] > — for all <& G (0, c] . (6) 

The truth of statement ([6]) can be checked by virtue of the following facts: 

| n 3 

• In the case of W a > c, it follows from © that statement (JS]) is true if and only if ip(X, 6, c) > — 

• In the case of W a < c, it follows from © that statement ^ is true if and only if 

^ X,6,W a > — 
71 

In 2 

max U(AT, 6, 0), 0(W a , ■&)] > — for all # G (W„, c] . (7) 

n 

The truth of statement ([7]) can be checked by making use of the following observations: 



In the case of 4>(W a ,c) < -^f- 7 we have 4>(W a ,-&) < — L for all z? G (W a ,c], since </>(Wa,$) is non- 
decreasing with respect to ■& G (W a ,c). It follows from ([2]) that statement ([7J is true if and only if 

In " 



1>(X,b t c) > 
In the case of <fi 

is non-decreasing with respect to 8 G (W a ,c). Thus, (j)(W a ,fl) < ^f- for all $ G (W a ,8*]. It follows 

from (J2J) that statement ([7]) is true if and only i: 
by a lower bound 9 which is extremely tight (for 
9 can be obtained by a bisection search method. 



In — In — 

In the case of <f>(W a ,c) > — l , there exists a 9* G (W a ,c) such that 4>{W a ,9*) = — since <j)(W a , 

In f 

from ^ that statement ([7]) is true if and only if ip(X,b,9*) > — In practice, 0* can be replaced 
by a lower bound 9_ which is extremely tight (for example, < 9* — 9 < 10~ 10 ). Such a lower bound 
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Therefore, through the above discussion, we have developed a rigorous method for checking the truth of 
([|5]). Based on this critical subroutine, we propose an efficient method for computing the lower confidence 
limit L for X > as follows. 

V Choose initial step size d > 77, where 77 is an extremely small number(e.g., 10~ ib ) . 

V Let F <- and a <- 0. 

V While F = 0, do the following: 

o Let st <— and £ 2; 

o While st = 0, do the following: 

★ Let £ <- I - 1 and d <- d2 e . 

* If a + d < X, then let b <— a + d. If (JSJ holds, then let st <— 1 and a <— b. 
-k If d < 77, then let st <— 1 and F «— 1. 

V Return a as the lower confidence limit L for X > 0. 



We call this algorithm as Adaptive Scanning Algorithm, since it adaptively scans the interval [0, X] to 
check the truth of ((6]). 

To compute the upper confidence limit U, we first need to establish a method to check, for a given 
interval [a, b] C [X, 1], whether the following statement is true: 

In 3 

For every v G [a,b], max \(p(X, v, i9), 0(^,1?) I/^ >W / >1 > — - for all $ G (0, z/(l - 1/)] . (8) 

n 

To check the truth of {8} without exhaustive computation, our approach is to find a sufficient condition 
for (|5J) so that the conscrvativeness of the sufficient condition diminishes as the width of the interval [a, b] 
decreases. For simplicity of notations, let c = max{a(l — a), 6(1 — b)} as before. As a consequence of 
®, we have ip(X,v,§) > tp(X,a,<&) for all v G [a, 6]. Since W a < W v < W b for 1/ G [a, 6] and 0(z,t?) 
is non-increasing with respect to z G (0,7?), we have (j)(W v , ■d)I{$ > w l/ } > 7?)I{ tf>W / b i. for v G [a, 6]. 

Hence, a sufficient condition for (j8]) is as follows: 

In 5 

max [<p(X,a,0), </>(W b ,#) I{# >Wb }] > — for all t? G (0,c] . (9) 
The truth of statement ([9]) can be checked by virtue of the following facts: 

• In the case of Wb > c, it follows from ((4]) that statement (J9j> is true if and only if (p(X, a, c) > 

• In the case of Wb < c, it follows from (j4|) that statement (j9|) is true if and only if 

N In I 
<p(X,a,W b ) > — 

77 

In - 

max|WX,a,tf), 0(W 6 ,tf)l > — for all 7? G (W&,cl. (10) 

71 

The truth of statement (jTOJ) can be checked by making use of the following observations: 

In — In — 

• In the case of <f>(Wb,c) < — we have </>(Wf,,#) < for all 7? G (Wb,c\, since </>(Wb,$) is non- 
decreasing with respect to t9 G (Wb, c). It follows from (fj| that statement (flU)) is true if and only if 

V (X,o >C )>y. 

In — In — 

• In the case of <fr(Wb,c) > — there exists a 0* G (W|,,c) such that ^(W 7 ;,, #*) = since <7i(Wi,,0) 

In 3 

is non-decreasing with respect to 9 G (Wb,c). Thus, (/3(W&,#) < for all 7? G (Wi,,0*]. It follows 

from ((3]) that statement (|10p is true if and only if y(X, a, 9*) < — s -. In practice, 6** can be replaced 
by a lower bound which is extremely tight (for example, < 0* — 9 < 10~ 10 ). Such a lower bound 
9 can be obtained by a bisection search method. 
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Therefore, through the above discussion, we have developed a rigorous method for checking the truth of 
(J5J). Based on this critical subroutine, we propose an efficient method for computing the upper confidence 
limit U for X < 1 as follows. 



V Choose initial step size d > 77, where 77 is an extremely small number(e.g., 10 ib ) . 

V Let F <- and b <- 1. 

V While F = 0, do the following: 

o Let st <— and £ «- 2; 

o While st = 0, do the following: 

★ Let I <- £ - 1 and d <- d2 e . 

-k If b - d> X, then let a <— b - d. If (J9j) holds, then let st «— 1 and b <- a. 

★ If g? < 77, then let st «— 1 and F «— 1. 

V Return 6 as the upper confidence limit U for X < 1. 



We call this algorithm as Adaptive Scanning Algorithm, since it adaptively scans the interval [X, 1] to 
check the truth of ©. 



3.2 Sequential Estimation of Mean 

In the preceding discussion, we have developed rigorous methods for constructing fixcd-samplc-size confi- 
dence intervals for the mean fi of the random variable X bounded in [0, 1]. Now, we are ready to construct 
a multistage sampling scheme which produces an estimator p, for fi such that Pr{|/i — /i| < e} > 1 — S, 
where e, 5 € (0,1). For this purpose, we consider a sampling procedure of s stages, with sample sizes 
m < 712 < • ■ • < n s chosen such that 

lnf . . ln£ 



In^ - — 2e 

At each stage with index £ £ {1, • ■ ■ , s}, we use the method described in Section [3.11 to construct a 
confidence interval (Lg, Ue) for \x in terms of X n{ = z - i = 1 ' and F„ f = ' =1 „' ' such that Pi{L e < 
fi < Ue} > 1 — Then, from Bonferroni's inequality, we have a confidence sequence {(£<>, fT^) , 1 < £ < s} 
such that Pr{Lf < /j < U#, I = 1, • • • , s} > 1 — S. By the inclusion principle, a stopping rule can be defined 
as follows: 

Continue sampling until there exists an index £ 6 {1, • • • , s} such that X nil — e < Le < Ue < -X^ + e. 
At the termination of the sampling process, take A"„ f with the corresponding index £ as the estimator (i 
for /i. 

According to Theorem [TJ the estimator fi for /i resulted from the above procedure ensures that Pr{|/2 — 
fJL\<e}>\-8. 



3.3 Confidence Region for Mean and Variance 

In many situations, it might be interested to infer both the mean \i and variance a 2 of X. For constructing 
confidence region for the mean \i and variance a 2 , we propose the following method. 

Theorem 4 Let 8 £ (0, 1). Define 

srf = :X< i/< 1, <0 < -1/), < -ln T , 0(W„,0) < -ln-rl, 

Ui/,1?) :X> 1/ >0, 0<1?< 1/(1-1/), ^(X>,0) < -ln|, 0(W„,0) < - In 1 1 
|_ 77, n J 

arwi 3>(X,V) = £/U.^. Then, Pr{(/j,cr 2 ) £ 0(Z,F)} > 1 - 5. 
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See Appendix [D] for a proof. The boundary of si is a subset of Ci U C2 U C3 , where 



Ci = 


{(",1?) 


C2 = 


{(",*) 


c 3 = 





4 no 



\jUv,#):X<v<l, Q<Q<W V , 0(W„,0) = i]n|J 



As a consequence of ©, y(X, 1/, $) is non-decreasing with respect to v. Hence, the points in Ci can be 
obtained by solving equation (p(X, u, #) = — In | for v € [X, 1) with a bisection search method. Note that 
(j)(W v ,d) is non-increasing with respect to $ € (0, W„) and is non-decreasing with respect to d € (W„, j). 
It follows that the points in C3 can be obtained by solving equation <p(W v , $) = ^ In | for 1? with a bisection 
search method. 

On the other side, the boundary of SB is a subset of D\ U Z?2 U -D3, where 

:X>j/>0, = i/(l-v)}, 

>iv>0, 0<tf<i ip(X, v,-B) = - In I 

■> 1/ > 0, VK^ < 1? < i 0(W W ,0) = - In I 
4 no 





= {M) 


D 2 


= {(M) 


D 3 


= {(M) 



I j{o,tf) :A->i/>0, 0<tf< W„, 0(W„,0) = -In I 
w (_ no 



As a consequence of (JXJ) , ip(X,v,'8) is non-increasing with respect to v. Hence, the points in D2 can be 
obtained by solving equation ip(X, v,{f) = — In | for G [X, 1) with a bisection search method. Note that 
4>{W„, •&) is non-increasing with respect to $ e (0, W v ) and is non-decreasing with respect to d G (W„, j). 
It follows that the points in £>3 can be obtained by solving equation <f>(W v , $) = r hi | for 1? with a bisection 
search method. 

Finally, we would like to point out that one can apply the same technique to develop confidence intervals 
and sequential estimation procedures for the mean and variance based on bounding the tail probabilities 
Pr {X > z} and Pr {X < z} by Bonnet's inequalities pQ or Bernstein's inequalities [2]. 

4 Conclusion 

In this paper, we have applied inclusion principle to develop extremely simple analytic sequential methods 
for estimating the means of binomial, geometric, Poisson and bounded random variables. Moreover, we 
have developed sequential methods for estimating the mean of bounded random variables, which makes 
use of the information of sample variance. Our sequential estimation methods guarantee the prescribed 
levels of accuracy and confidence. 

A Derivation of Stopping Rules 

For simplicity of notations, define 5? = {1, ■ • • , s}. 
A.l Derivation of Stopping Rule A 

We need some preliminary results. As applications of Corollary 5 of j5], we have Lemmas Q] and [2] 
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Lemma 1 Let fx G (0, 1). Let m £ JV and e 6 (0, 1 — fx). Then, 
[yyi \j n)s 

X n < /i + — for all n € Jf \ > 1 - exp {mJ^^{^, + e, fi)) . 

Lemma 2 Let \x G (0, 1). Let m G jY and e G (0, //). Then, 

Pr i I n > jU — ( m v n ) £ y or ^ n g ^/ 1 ^ _ f m ^ B fn _ £] ^A) _ 
I » J " 

Lemma 3 Let y, r 6 (0, 1]. Then, ./#b (m + r (z/ — A*) > A*) * s non- decreasing with respect to [i G (0, j/). 

Proof. From the definition of the function we have that ^b(z,h) = zln^ + (1 — z) In for 

z G (0, 1) and /x G (0, 1). It can be checked that = m and ^b(*,m) = _jLziL_ for z e ( 0; l) 

and /j G (0, 1). Now let 2 = (J, + r (y — fi). Since fj, G (0, y), it follows that z G (0, 1). Hence, 

(fj, + r (y - n) , fi) _ _ r )i n .^ t ( 1 ~ z ) ■ 



z(l-M) A*(!-m) 



> (l-r)Qi-z) /j-z = r(At-z)fa-l) >Q 



This completes the proof of the lemma. 



□ 



Lemma 4 Let y G [0, 1) and r G (0, 1]. Then, (A 4 — r (m — 2/) i A 1 ) * s non-increasing with respect to 
A* G 

Proof. For simplicity of notations, let z = /i — r (fx — y). Note that 

b (n-r (n -y) ,n) = _ ^ fi(l - z) , z - /i 



< (1 - - z) /j - z _ r(z - < 
z(l - /j) - /i) /z(l - \£)z ~ 

This proves the lemma. 

□ 



Lemma 5 For n G JV and i G 5? , define 

forX n = 0. 



Li 



Then, Pr{L e n < fj, for all n G JT) > 1 - £ for I G J? 7 . 

Proof. First, we need to show that is well-defined. Since L n = for X„ = 0, L l n is well-defined 
provided that L^ exists for < X n < 1. Note that lim^^B^ + „ (j/ — = > In ^ for 

y G (0,1]. This fact together with Lemma [3] imply the existence of L l n for < X n < 1. So, L l n is 
well-defined. From the definition of L„, it can be seen that 

{fi < L £ n , X n = 0} = {fx < X n , J£b (ft + — ^— (In - A*), A* J < — In X„ = 
L \ n V mi J mi Zs 

{/j < Li, < X n < 1} C |/x <X n , J? B (ft+ " (X„-/x),/i) < — ln^-, < X„ < 1 

I V n V ra; / to^ 2s 
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This implies that {/x < L e J C { M < X n , Jt^ + ^(I„ - fx), /i) < ^ In £}. 
Next, consider Pr{L^ < /x for all n € for two cases as follows. 
Case A: fi me < ^. 
Case B: /j m ' > 

In Case A, there must exist an e* £ (0, 1— /i] such that (a* + £*, /x) = ^- In Note that ^b(^+£, A 4 ) 
is decreasing with respect to e G (0, 1 — /x). Therefore, from the definitions of and e* , we have that 
{n_< ill C {/x < J? n , ^ B (/x + 7^l(Xn - H),fi) < C { M < X„, ^(T n - M ) > £*} C 

{!„ > + (" V ^) E " }, This implies that {L^ < /x} D {A„ < /x + (raV " lf)e '' } for all n £ J/. Hence, 
{L e n < (i for all n £ JT) D {X n < fi + ( " v " <)£ * for all n £ JV}. It follows from Lemma ED that 

Pr{L£ < /x for all «£/}> Pr{A„ < /x + (" v ^) £ * for all n e ^} > 1 - exp (m^ B (> + £*, M)) = 1 - s 
for all fey. 

In Case B, we have {/x < A„, ^ B (/J + ^^(^n ~ £*)>/•*) < ^7 ln ^} = it 1 < hi/x < ^b(m + 
j&^(X~n - /x),/x) < ^ln^} = 0. It follows that {/x < L e n } = for all n £ JV . Therefore, Pr{££ < 
\x for all n £ JV} > 1 — S n e,/K P f {M < ^1} = 1 f° r au ^ G which implies that Pr{if t < fi for all n £ 
.yf} = 1 for all fgy. This completes the proof of the lemma. 

□ 



Lemma 6 For n £ JV and i £ 5? , define 

ut = J sup [u £ (X n , l) : ^ n (v--£-(v- X n ) , u) > £ In ±} forX^K 1, 
\l forX n = l. 

Then, Pr{E7* > /x for all n £ JT) > 1 - ^ f° r al1 Z^-?- 

Proof. First, we need to show that is well-defined. Since E/~ = 1 for X n — 1, [/„ is well-defined 
provided that C/^ exists for < X n < 1. Note that lim,,^ ^&(y — „ v " ra{ ~ V) > v ) = - > m7 m 5s 
for yy £ [0,1). This fact together with Lemma [4] imply the existence of for < X n < 1. So, [/^ is 
well-defined. From the definition of J7„, it can be seen that 

{n > U e n , X n = l}= \n>X n , Jf B U ^— (n-X n ),ft) < — ln^-, I B = lj =0, 

{/i>C/^, < I n < 1} C \ n >X n , ^ b (m £—(ji-X n ),f>t) <— In— , 0<X n <ll. 

L \ nv mg ) me Zs J 

This implies that {p > U 7 {} C { M > X„, ^ B (^ - " ^„), /i) < ^ In £}. 

Next, consider Pr{f7^ > for all n € -vK} for two cases as follows. 
Case A: (1 - ^ < ±. 
CaseB: (1 - ^ > 

In Case A, there must exist an e* £ (0, fj] such that (a* ~ £ *> Z 1 ) = ^7 m Note that ^b(h — e , I 1 ) 
is decreasing with respect to e 6 (0, fi). Therefore, from the definitions of {/„ and e* , we have that 
{^> ^} C {/x > X n , J? B (» - ^(/x - X„),/x) < ^hi A} C {/i > X„, ^(/x - X„) > e*} C 

{!„ < /x - ( " v " f)E * }. This implies that {U^ > /x} D {A„ > /x - (" V ^K ]. fo r a n n e Hence, 

{C/^ > /x for all 71 e ,yV} 2 {A„ > /x - ( " v " <)e * for all n £ JT}. It follows from Lemma M that 

Pr{C/^ > a* for all n £ jV} > Pr{X n > /x- ( " v " t)£ '' for all n £ J/} > 1 — exp {m t JK^{n - e*,/x)) = 1 - ^ 
for all fey. 

In Case B, we have {/x > A„, ^ B (/i - - ^n),/i) < ^In^} = {m > A„, ln(l - /x) < 

^b(a* ~ - A n ),xt) < ^ln^} = 0. It follows that {xx > U,{} = for all n £ jY . Therefore, 

Pr{C7^ > fi for all n £ > 1 - E ra e^K Pr {M > = 1 for all £ £ .Y, which implies that Pr{D r / 1 > 
/j for all n £ ,jV} = 1 for all £ £ . This completes the proof of the lemma. 
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Lemma 7 



{X n - e < L e n < U l n < X n + e} 

[X n +e ,X n + e < — In — , [X n - e + — ,X n - e < — In — 

\ n V rat ) me Is \ n V me ) me Is 

for all n G ,jV and I G . 

Proof. From the definitions of and L l nl it is clear that 

Xn - e + — ^,X„ - e) < — In A X n - e < oj = {X n - e < 0, X n -e< L e } 
n V mi J 2s J 

for n G Ji . By Lemma [3] and the definition of L l nl 

X n ~e + — ^,X„ - e) < — In A, x n - e > o) = {0 < X n - e < L l n } 
n V me J me 2s J 

for n£/. It follows from {TTJ) and (JT5J) that 

(X n -s< L e } = I^B fe, - £ + " £ ,X B - e J < — hi A 
L \ nV me J rag 2s 

for n 6 ,# and ^ G ^. 

On the other hand, from the definitions of and it is clear that 

X„ + e- — ^,X„ + e) <— In A X n + £>l\ ={X n + e>l, X n + e>U l n } 
n V me J mi 2s J 

for n G o/K. By Lemma [4] and the definition of U„, 

? B (x n + e - -^,X n +e) < —In A X n +e < l\ = {1 > X n + e > U l n ] 
\ ?i V me J me 2s J 

for n G t/K. It follows from flHI and (fT5l) that 



{X„ + e > [#} = (^B [X n + e ^— , Z„ + e ) < — In A 

for ?i G jY and ^ G J^. Combining (|T3]) and (|L6|) completes the proof of the lemma. 



Lemma 8 Let < e < \ and < r < 1. Then, 

./#b (y + £ — ?'£, 2/ + e) < ^#b (y — s + re, y — e) /or y G 
./# B (y + e - re, y + e) > ^ B (y - e + re, y - e) for y G 



2' 
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Proof. First, we need to show (fTTj) . Note that 0<e<i<y<l. For y > 1 — e, we have < y — e < 
y — e + re < 1 and Ji^, (y + e — re,y + e) = — oo < .-#b (y — £ + re,y — e). Thus, to show (fT?)) . it suffices 
to show .^#b (y + e — re, y + e) < .<#b (j/ — £ + re, y — e) for i < y < 1 — e. Let 9 = y + e, z = y + e — re 
and $ = y — e, w = y — e + re. A tedious computation shows that 

(y + e-re, y + e) dJ?B (y - £ + re,y - e) 



de 

{ ' z{\ - 9) w{\ - d) 



c)s 



— re 



1 



<(l-r) 



(1 - z)ti(l - w) 



1 



tf(l-tf) 
1 



z(l - 9)w(l - ??) 
r 2 £ 2 (2y-l)[(r-3)y 2 -(l-r) £ 2 ] 
(1 - 0)(1 - tf)(y 2 - e 2 )[2/ 2 - (1 - r) 2 £ 2 ] 



1 - 6 
< 



for 5 < 2/ < 1 — £• Using this fact and the observation that (y + £ — re, i/ + e) < (y — £ + re,y — e) 
for e = 0, we have that ^#b (y + £ — re, y + e) < .-#b (y — £ + re, y — e) for i < y < 1 — e. Thus, we have 
shown (fT7|). 

Making use of (JTTJ) and the relationship that ~#b(z, 0) = ^#b(1 — 2j 1 — 9) for 2 G [0, 1] and 9 G (0, 1), 
we can conclude (jT5J) . 

□ 

Making use of Lemmas [3 |8] and the fact that ^(-b(z, 9) = ^b(1 — z, 1 — 9) for z € [0, 1] and 9 G (0, 1), 
we have the following result as asserted by Lemma |9l 



Lemma 9 

(X n - e < Li < Ut < X n + e} = 
for all n G ,jV and I G . 













2 



ne 1 



n V mi ' 2 



1 , s 
+ e < — In — 

mi is 



Lemma 10 Define C n = max£ £ y L l n andU n = min^g,^ f/„ /or n G ^/K. Then, Pr{£ n < /j, < U n for all n G 
^} > 1-6. 

Proof. Recall that Pr{L^ < fj, for all n G Jf} > 1 — j- for £ G J^. By Bonferroni's inequality, we have 
that Pr{L^ < /x for all n G ^ and all I G ,5^} > 1 — I, which implies that 



Pr{£„ < /j, for all n G ^K} > 1 



(19) 



On the other hand, note that Pr{[/^ > /.t for all n G -vV} > 1 — j- for £ G J^. By Bonferroni's inequality, 
we have that Pr{[/^ > fj, for all n G JV and all £ G S?} > 1 — f , which implies that 



Pr{W„ > /i for all n G -yV} > 1 - - 
Combining (jT9]) and (|20|) proves the lemma. 

Lemma 11 Vx{X n — e < C n < U n < X n + e for some n G j¥ } = 1. 



(20) 
□ 
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Proof. By Lemma [HI and the definition of C n and I4 n , we have 



Pr{X n — e < C n < U n < X n + e for some n £ ^V} 
> Pr{X ms - e < < 17^ < X m , + e} 



Pr 



X r 



X r 



1 , * 
< In — 

mi 2s 



As a consequence of the assumption that m s > 

2* - 



— In T7- 1 = 1 , from which the lemma follows 



2s- 



we have Pr 



' 2 



|i-X„| + £ )< 



□ 



Now we are in a position to prove that stopping rule A ensures the desired level of coverage probability. 
From Lemma[9], we know that the stopping rule is equivalent to "continue sampling until {X n — e < L n < 
U n < X n + e} for some t £ =5^ and n £ c/K" . We claim that this stopping rule implies that "continue 
sampling until {X n — e < C n < I4 n < X n + e} for some n £ --/V" . To show this claim, we need to show 

U [j {Xn~e<L n < Ui <X n +e}C [j {X n - e < C n < U n < X n + e}, 

ley ne.jY n&jY 

which follows from the fact that \J ees ?{X n - e < L l n < < X n + e} C {X n - e < C n < U n < X n + e} for 
every n £ JY '. From Lemma 1 1 1 1 we know that the sampling process will terminate at or before the s-th 
stage. It follows from Lemma [TUl and Theorem [1] that Pr{|A" n — [i\ < e} > 1 — S. 



A. 2 Derivation of Stopping Rule B 

Define function Mb(-, ■) such that 

( 9(z-8) 2 



M B (z,t 



2(z+29)(z+29-3) 

— OO 



forz 6 [0,1], 6 £ (0,1), 
for z £ (-oo, oo), 9 £ (0, 1). 



We have established the following result. 
Lemma 12 



X n - - 



Mi 



IIS 



-i 2 



— £ 



3(n V mi) 







g- 


— — x ri 




2 



1 

> — 

- 4 

ns 



2 2 



n V mi ) 2 ln 2 f 



n V mi ' 2 



X r 



1 , 8 
e | < — ln — 

77l£ 2s 



for all n £ <yK and ^ £ {1, • • • , s}. 



Proof. Let y £ [0, 1], r £ (0, 1], a £ (0, 1) and m £ JV . For simplicity of notations, let 9 



e, z — 9 — re and w = z+2e . Then, 9 — z = re, w = ( 
\ - {\ - w) 2 > 0. Moreover, 9 £ (0, 1), z £ [0, 1]. It follows that 



f and w(l - w) = (| - w + \)[\ - (| - w)] 



. j , r\ \ bi a 

M B {z,6) < 

m 



(ref 



2 , 1 
> — ln- 



2 — (| — w) 2 m a 



- - w\ > - 



1 m(reY 



21na 



which implies that 





1 




re 




^2 


-e-\ 


' Y 



> 



1 m(reY 
4 + 21na 



Mi 





i 


1 


1 


a- 


2 _2/ 


+ e-re, - - 


2" y 







£ < 



lna 



m 
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This proves the lemma. 



□ 



As a consequence of Lemma 1121 stopping rule B is equivalent to the following stopping rule: 
Continue sampling until Mb ( § — \ \ — X n | + e — - " ? £ Ttg , \ — \ \ — X n | + e) < ^- In ^ for some inte- 



gers n£ / and I e {1, • • • , s} 

As a consequence of Massart's inequality, 





1 — 




Mi- 


77 — X n 
2 


+ e 



71 1 



1 



C 



— X n 



n V mi ' 2 
ne 1 

, V mi ' 2 



1 , <* 
< — In — 

me 2s 



- e — 







— — X-n 

2 


+ e) < — ln^-l 


J mi 2s J 



for all n G ^ and I e {1, • ■ • , s}. 

Thus, by a similar method as that used in Appcndix lA.ll to justify that stopping rule A guarantees the 
desired level of coverage probability, we can show that stopping rule B also ensures that Pr{|A n — ji\ < 
£}>1-S. 

A. 3 Derivation of Stopping Rule C 

We need some preliminary results. 
Lemma 13 



y 



rey y 



l + e 1 + e'l + c 



> 



rey y 



1 — e 1 — e' 1 — e 



for y, r € (0, 1] and e € (0, 1). 

Proof. In the case of y > 1 — e, we have ^#b(t^ t^-, t^—) = —00 < ^£b(-tt — I- tt^i tt-)- Therefore, 

it suffices to show the lemma for the case that < y < 1 — e. For simplicity of notations, define H(e) = 

^B(^ ;T ^).Then, 

y(l + re)' 



H(e) 



1 



1 



1 + T^v 2/(1 + r£ ) , ft % 
In +— 1 — ln(l + re). 



1 + tziv £ 



l + e 



By virtue of Taylor expansion series 

y . 1 



ln(l + re) = X:(-l) m V' 



In 



i-y 



1 + kziy £ 



1+1 1 



(i-vY 



1 - ry 
1-2/ 



and a lengthy computation, we have 

(1 - e 2 )[H(e) - H(-e)} = 2^ C(r,y,2k + 1) (1 - y)~ 2k e 



■2fc„2fc+l 



k=l 



where 

C(r,y,£) 



r e-2( 1 _ y y-i /(i_^-i (l- ry )(l-y) 
£ 1-2 



1-2 



1 



r\\-yf + \-{l-ry) 
for I > 2. A tedious computation shows that 



(1 - r)y 
1-1 



" 2 (l-^- 2 + i-(i-r y y 

1(1- tf )'-i + !-(!- ryf 



dC(r,y,£) y \ t-i,* si-i . -, ,-. . ry(l—y) r M , . » ,., ye-2l 
^ — - = |3Tr +l-(l-ry) j + e _ 2 [ r ^-y) +l-(l-ry) j 



> 0. 
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But C(r,y,£) = for r = 0. Thus, C(r,y,£) > for all r £ [0,1] and y £ [0,1]. This proves that 
H(e) — H(—e) > for y, r £ (0, 1] and e G (0, 1). Thus the lemma is established. 



Lemma 14 



<Li < Ui < D (jf n >0, ^ B ( ^ ("l + ^^V^ < -ln^ 



1 + s J V 1 + e V nV mi J 1 + e J me 2 



for &\\n £ ,yV and £ G N. 

From the definitions of ./#b and C/„, it is clear that 



' B 



1 — e \ nv m£ / 1 — e J mi 1 J |_ 1 — e 

for n £ JV . By Lemma 2] and the definition of U, 



' B 



□ 



1 + e 1 — e J [ + nVm^/l + e/ me 2 

for all n £ ,yV and £ G N. 

Proof. From the definitions of and L„, it is clear that 

fe f ^ f 1 + -Oh) , ^) < - 14, = 0} = ( x « = °' rr- * L ^ ^ 

\ 1 + £ \ nV mi J 1 + e / 2 J [_ 1' 

for n £ jY . By Lemma [3] and the definition of L„, 



1 + £ \ nV m£ J ' 1 + £ J mi 2 ' J \ 1 + £ 

ioT n £ J / . It follows from (gTl) and that 

^l^ufe&fl^y^ain^ (23) 



( x _ m\ Xn_\ ± ln St Yn+£ >A = !x n +£>l, -^>U<\ (24) 



^(l-^),^)<-^,£<X n + £<l) = {£<X n + £<l,^>Ui\ 

1 — £ \ nV mi J I — £ J mi 2 J 1 — e J 

(25) 



for n G JV . It follows from {H} and (US]) that 

7«>0,ufe(^-(l— =f-),^)<J-4}c{^L>^} (26) 
\ 1 — e \ nV mi J I — e J mi 2J 1 1 — e J 

for n £ jY and £ £ N. Finally, the proof of the lemma can be completed by combining (|2"5|) . (|2"6"|) and using 
Lemma 1131 

□ 



Lemma 15 Define C n = sup^ eN L n and U n = inf^gN f/„ for n £ jV . Then, Pr{£„ < // < U n for all n £ 
JT}>1- 8. 
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Proof. Recall that Pr{L^ < \i for all n G JV} > 1 - ^ for < e N. By Bonferroni's inequality, we have 
that 

Pr{i£ < fj, for all n G jY and I = 1, ■ ■ ■ , k} > 1 - £ 
for any fc G N. By the continuity of the probability measure, we have 

Pr{L^ < [i for all n G JV and /gN} = lim Pr{L^ < \i for all ne / and ^ = 1, ■ • ■ , k} 

> i- lim gkj£ = i_^ ; 

»oo 2 2 

which implies that 

Pr{£„ < |U for all n G ^K} > 1 - -. (27) 

On the other hand, note that Pr{J7/ l > /i for all n G o/K} > 1 - 4f for £ G N. By the continuity of the 
probability measure and Bonferroni's inequality, we have that Pr{f/^ > [i for all n G <yf and all £ G N} > 
1 — |, which implies that 

Pi{U n > m for all n G ^T} > 1 - -. (28) 
Combining (|2"7) and (f2"5| proves the lemma. □ 



Lemma 16 Pr{y^ < C n <U n < ^ for some n G ,yV} = 1. 

Proof. By the definition of C n and U n , it is sufficient to show that Pr{^^f < L f mf < U f mi < -jz^r for some £ G 
N} = 1. From Lemma [T4l it can be seen that 

Pr ( < L i < [/* < for some £ G n) 

> Pr \X mi > 0, ^ B [ X mi , ) < — In % for some £ G N } . 

L V l + e/ m £ 2 J 

This inequality and Bonferroni's inequality imply that 

p r (Xnu < L i <u i < ^nu for SQme t N | 

> lim Pr{X ro< > 0} + lim Pr fe„ f , ) < — In ^ 1 - 1. 

Since /j, > 0, it follows from the law of large numbers that lim^oo Pr{X mf > 0} = 1. To complete the proof 
of the lemma, it remains to show that lime^^ Pr |*#b ^m f ,^7") — ^^if} ~ 1- This is accomplished 
as follows. 

Let < r] < 1. Noting that ^ In 4f is negative for any £ > and that ^ In ^ -> > .-# B (??M, t+i) 
as £ — > oo, we have that there exists an integer n such that J(-&(j)pt, y^) < ;^Tn4f for all £ > k. For £ 
no less than such k, we claim that z < if ./#b(z, yjj) > ^7 l n if an d 2 £ [0, 1]- To prove this claim, 
suppose, to get a contradiction, that z > 77//. Then, since ^b(z, is monotonically decreasing with 
respect to z G (0,1), we have Jis(z, j^) < ^£b(ti^, ^p) < ln ^, which is a contradiction. Therefore, we 

have shown the claim and it follows that {Jt^(X mi ,, > lk~ t ^ n ^} — {Xm e < VI 1 } f° r £ > k. So, 



1G 



for large enough £, where the last inequality is due to the multiplicative Chcrnoff bound. Since me — > oo 
as I — > oo, we have lirn^oo Pr |^#b \X mt , -j-Sr) < ^- In } = 1- This proves the lemma. 

□ 

Now we are in a position to prove that stopping rule C ensures the desired level of coverage probability. 
From Lemma [TJJ we know that the stopping rule implies that "continue sampling until < L„ < < 
f° r some I G N and n G Jf" . We claim that this stopping rule implies that "continue sampling until 
{j^ < C n < U n < j^t) f° r some n G ■ To show this claim, we need to show 

U U {^<4<^<^}= U {^<A,<u„<^}, 

which follows from the fact that \J eeN |^ < < Ui < ^| C |^ < £ ?l < W n < 2k | for every n e 
From Lemma \16\ we know that the sampling process will eventually terminate. It follows from Lemma [T5l 
and Theorem □ that Pr{|X n - fx\ < efi} > 1 - S. 

A. 4 Derivation of Stopping Rule D 

We need some preliminary results. As applications of Corollary 5 of [5], we have Lemmas 1171 and 1181 
Lemma 17 Let 9 G (1, 00). Let m G jV and e > 0. Then, 

Pr \ X n < 8 + ± '- for ollne/ >1- exp (mJZ G (0 + e, #)) • 



Lemma 18 Let 8 G (1, 00). Let m G jV and e G (0, 9). Then, 

[ [77 Y/ 777 )c 

Pr \X n > 8 - ± '- for alined} > 1 - exp (mJZ G (6 - e, (9)) . 



Lemma 19 Lei y > 1 and < r < 1. Then, ^ G (9 + r(y — 9), 0) increases with respect to 9 G (1, y). 

Proof. For simplicity of notations, let z = 9 + r (y — 9). It can checked that d -^o^,s) _ m ^"^j an( j 
_ 8=z m By the chain rule of differentiation and the inequality ln(l + x) < x for x > — 1, we 

have 

d^ G (9 + r(y-8),8) 9-z 9(1 - z) 

= 1 — r m 

88 0(1-8) 1 1 z(l-9) 

9-z (1 - r)(8 - z) _ (8 - z)[z - (1 - r)9] _ (9 - z)ry 

~ 8(1 - 8) z(l-8) ~~ 8(1 -8)z ~~ 8(1 - 8)z > ' 

This proves the lemma. 

□ 



Lemma 20 Let y > 1 and < r < 1. Then, y^ G (9 — r(8 — y),8) decreases with respect to 9 G (y, 00). 
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Proof. For simplicity of notation, let z = 9 — r (9 — y). It can checked that d -^^ z ' e > = In z Jy\ and 



dJ( G {z,B) _ 



9(1-2) 



Hence, 

d^ G [e-T[e-y),e) 
de 



= (l-r)ln 



z{l-$) z 



< 



(l-r)(z-0) z 



,1 - z) 6>(1 -0) - 6>(1 - z) 0(1 
(9 - z)[(l - r){9 -l)-(z- 1)] (9 - z)r(l - y) 



e(e-i)(z-i) 



e(e-i)(z-i) 



< o. 



This proves the lemma. 



Lemma 21 For n € JV and £ g 5? , define 



□ 



L 



\n£{ve(l,X n ):^ G (v+^(X n -v),v)>±ln£;} forX n >l, 
1 forX n = l. 



Then, Pr{L £ n < 9 for all n £ Jf} > 1 - £ for I £ 5? . 

Proof. First, we need to show that is well-defined. Since L e n = 1 for X n = 1, L l n is well-defined 
provided that L e n exists for X n > 1. Note that lim^ ./# G [v + raV "„ g (y ~ ^) i ^) = > ^- In ^ for 
y € (l,oo). This fact together with Lemma [Ti?l imply the existence of for X n > 1. So, L l n is well- 
defined. From the definition of L l nl it can be seen that 

+ ^—(x n -e),e) <— in A, x n = i) = 0, 

T1 \ I A 

+ — (X„-0),0 < — In— , X n > 1 

n V J vug Is 

-j—(Xn - 0),0) < — In f }. 



{o < L' n , x n = 1} = |0 < x n , 
{0<£l, X„>i}c (^<X n , 



This implies that {6> < L e n } C {9 < X n , J( G ( 

Next, consider Pr{Lf, < 9 for all n £ <yf}. Since 



lim 

t— »oc 



*in--(*-i)m — 



lim 

t— »OQ 



tin- 



t - 1 



ln(0-l)+ln(f-l)-tln- 



1 



there must exist an e* > such that j^q (0 + e*,0) = ^hi^. Note that ^q(9 + e,0) is decreasing 
with respect to e > 0. Therefore, from the definitions of L n and £*, we have that {9 < L^} C {9 < 
Xn, J* G {0 + ^(X„ - 6),6) ^In^} C {0 < X n , ^{X n - 0) > e*} C {X n > 9 + InX^lil}. 
This implies that {L l n < 9} D {X n < 9 + iliX^ilil} for all n £ ,yV . Hence, {L l n < 9 for all n £ ^K} 2 
{X n < + (" Vm ^ £ * for all n e }. It follows from LcmmaQIlthat Pr{Lf l < for all ne/}> Pr{X„ < 



^ . (iiVm ( )6* 
n 

the lemma. 



for all n £ J^} > 1 - exp {mi^o,[9 + e* , 9)) = 1 - ^- for I £ 5? . This completes the proof of 



□ 



Lemma 22 For n £ jV and I £ 5?, define = sup{^ £ (X n , 1) : Jt( G (v - ^^-(v - X n ),v) > ^;ln^} 
Then, Pr{l^ > 9 for all n £ Jf} > 1 — ^ for I £ . 
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Proof. First, we need to show that U* is well-defined. Note that lim v ± y j% G (y — n ™ {y — y) , v) = > 
^- In ^ for j/ G [1, oo). This fact together with Lemma [20l imply the existence of U^. So, is well-defined. 
From the definition of U*, it can be seen that {6 > U*} C {6» > X n , JZ G (6- ^_(6-X n ), 0) < ^ In £}. 



Next, consider Pr{[/„ > 9 for all n G JY} for two cases as follows. 



Case A: 9~ me < 
Case B: 9~ me > 2 



2s 

In Case A, there must exist an e* G (0, 9—1] such that Ji G (0 — £*, 6) = hi ^. Note that „#g(0^ £ , 
is decreasing with respect to e G (0,9 — 1). Therefore, from the definitions of and e*, we have that 
{^> Ui) c{9> X n , Ji G (9 - -^{6 - X n ),6) < ih^} C {0 > X„, ^(0 - X„) > £ *} C 
{X n < 9 - ( " v 7 )£ " }. This implies that {E# > 9} D {X n > 9 - ili^iK} for a n n G ^. Hence, 
> 6» for all n G ^f} 3 {A n > - ( " v, ? " g)E * for all n G .vK}. It follows from Lemma HU that 
Pr{U* > 9 for all ne/}> Pr{I n > - (» V "^) E * for all n G ^K} > 1 - exp (m^ G (0 - e*,6)) = 1 - ^. 

In Casc_B, we have {0 > I„, .# G (0 - ^(0 - X„),0) < ^ In A} = {fl > x n , In \ < JZ G {9 - 
;^(0 - I„),f) < ^ln^} = 0. It follows that {9 > U l n } = for all n £ ,yY. Therefore, Pr{l^ > 

9 for all n£/}> l-£ne^ P^ 61 ^ ^n} = 1 for £ G J^, which implies that Pr{U^ > 9 for all rt G .yY} = 1 
for £ G 5? . This completes the proof of the lemma. 

□ 



Lemma 23 

{(1 - e)X n <L e n <U e n <{l + e)X n } 

l+e-^^)x n ,(l + e)X n ) < J( G ( (l-e+^-)x n ,(l-e)X„) < ^ 

/or all n e yY and ley. 

Proof. From the definitions of and L n , it is clear that 

l-e+-^-)x n ,(l-e)X n ) < — In A (1 - s)X„ < lj = {(l-e)X„ < 1, (1 - e)X n < L e n } 
n V mt I I mi Is J 



for n G JY . By Lemma [T9l and the definition of L l n , 



(29) 



1 - e + -^—] , (1 - e)X n ) < — In A, (1 - e)A„ > 1 J. = {1< (1 - e)A„ < L^} 
for n G JY . It follows from (HU) and (f30l) that 



{(1 - e)X n <Li} = \^ G [[l-e+ ) A„, (1 - e)X n ) < —In A I, 



n V mi J J rag Zs 

for n G jY and l € 5^ . By Lemma |2T)1 and the definition of U^, 

l + e--^-)x n ,(l+e)X n ) <— lnf^ 



for n G c/K and £ G Combining (j3"Tj) and p2p completes the proof of the lemma. 



□ 



Lemma 24 Jt G ((1 -e + rs)y, (1 - e)y) < ^ G ((1 + e - re)y, (1 + e)y) /or e G (0,1), r G (0,1] and 

y> i. 
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Proof. In the case of 1 < y < j^, we have ^G{{\ — e+re)y, (l—e)y) = — oo < ^Q((l+e — re)y, (l+e)y). 
Therefore, it suffices to show the lemma for the case that y > j^. For simplicity of notations, let 
v = (1 — e)y, $ = (1 + e)y, z = (1 — e + re)y and w = (1 + e — re)y. Note that { z , v ) = -^G (w, for 
e = and 



9e 



0e 



y(l - r) In -)- + y— + y(l - r) In -i- -( + y 



z(l -i/) - 
= (1 — r)y In — — — — ^ — rey 



< (1 - r )i/ 



z(l - i/)uj(1 - #) 
1/(1 - - w) 



io(l-i?) a t?(l-t9) 
1 1 



1 



z(l - i/)tu(l - <&) 
(re) 2 [r-3-(l-r)e 2 }y(2y-l) 



(l + e )(l-0) (1 -£)(!-!/) 
1 1 



(l+ e )(l-0) (1-eXi-!/) 
<0, 



(l-^)(l-tf)(l-e 2 )[l-(l-r) 2 £ 2 ] 
where the last inequality is a consequence of r £ (0, 1] and y > 1. This proves the lemma. 



Making use of Lemmas l23l and l24l we have the following result. 
Lemma 25 

ne 



□ 



{(1 - e)X„ <L e n <U t n <(l + e)X n } = 
for aline jV and I £ 5?. 



1+e- 



n V 



X n , (1 + e)X n ) < — In A 
mi 2s 



By a similar argument as that for proving Lemma [26l we have established the following result. 
Lemma 26 Define C n — ma,X( e y L n andM n = min^g^ forn £ jV . Then, Pr{£„ < fi < U n for all n £ 

.yK}>i-5. 

Lemma 27 Pr{(l - e)X n < C n < U n < (1 + e)X n for some n £ JV} = 1. 

Proof. By Lemma |2~51 and the definition of C n and U n , we have 
Pr{(l - e)X n <C n <U n <(l + e)X n for some n £ =yf } 

{1 5 
,# G (X ms , (1 + £ )X ma ) < — In — 

As a consequence of the assumption that m s > in(i+e)- E > we nave P r {^G(X ms , (1 + e )-^m s ) < 
— ln = 1, from which the lemma follows. 

□ 



Now we are in a position to prove that stopping rule D ensures the desired level of coverage probability. 
From Lemma [231. we know that the stopping rule is equivalent to "continue sampling until {(1 — e)X n < 
L l n < < (1 + s)X n } for some f 6 y and n £ Jf". We claim that this stopping rule implies that 
"continue sampling until {(1 — e)X n < C n < (1 + e)tl n < X n } for some n £ JV\ To show this claim, we 
need to show 

U \J{(1- e)X n < L e n < C# < (1 + e)X n } C (J {(1 - £ )X„ < C n < U n < (1 + e)X n }, 

which follows from the fact that U tey {(l-e)In < L e n < Ui < (l+s)X n } C {(l-e)X„ < £„ < W„ < (l+e)X„} 
for every n £ From Lemma [2T1 we know that the sampling process will terminate at or before the s-th 
stage. It follows from LemmaHiand Theorem [Q that Pr{(l - e)X n < 9 < (1 + e)X n } > 1 - S. 
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A. 5 Derivation of Stopping Rule E 

We need some preliminary results. 

Lemma 28 Let y > and < r < 1. Then, ^#p(A + r(y — A), A) increases with respect to A < y. Similarly, 
^#p(A — r(A — y), A) decreases with respect to X > y. 



Proof. Note that ^ P (z, A) = z-A + zln|. It can be checked that M|feAJ = i n A an d d^{z,\) = *_ L 
For simplicity of notations, let u = A + r (y — A) and u = A — r (A — y). By the chain rule of differentiation, 

p (A + r (y ~ A) , A) _ u - A ^ _ ^ ^ u > m- A _ ^ _ ^ u- A = r(u - A) > Q 



<9A A A _ A A A 

fr^P (A - r (A - y) , A) = (1 _ r)ln A + v-\ < A - « _ A - u = (t; - A)y < Q 

c*A tj A — v A uA ~~ 

This proves the lemma. 



□ 



Lemma 29 ./#p + e — re, y + e) > .<#p (y — e + re, y — e) for e > 0, y > and r G (0, 1] . 

Proof. In the case of < y < e, we have ^p (y — e + re,y — e) = — oo < ^#p (y + e — re, y + e). 
Therefore, it suffices to show the lemma for the case that y > e. For simplicity of notations, let 9 = 
y + e, $ = y — e, z = y + e — re and w = y — s + re. Note that 

0,0) dJtp{w,-d) 



de de 



,1 1\ ,„ , , zw ( 1 1\ , . f zw \ (re) 2 (3-r) 

= re (1 - r) In — > re (1 - r) 1 ) = ^4 > 0. 

\d 9 J y ' 6d ~ \$ 9J K '^9-0 J 9d 

The proof of the lemma can be completed by making use of this result and the observation that ^#p(z, 9) = 
^p{w, d) for e = 0. 



□ 



As applications of Corollary 5 of [5] , we have Lemmas 130] and I3T1 
Lemma 30 Let A G (0, oo). Let m G j¥ and e > 0. Then, 

Pr { X„ < A + ( m V n ) £ y or a // n G jA > ! _ exp ( TO .^ p (A + e, A)) . 



Lemma 31 Let A G (0, oo). Let m G jY and e G (0, A). Then, 

Pr < X n > A — ^ ^ for all n G \ > 1 - exp (m^ P (A - e, A)) . 



Lemma 32 For n G JV and <6N, define 



'inf {„ G (0, X n ) : ^ P (i/ + ^ (X„ - v) ,v) > £ In §} /orI n > 0, 
forX n = 0. 



Then, Pr{L e n < A for all n G Jf } > 1 - f /or ^ G N. 
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Proof. First, we need to show that L n is well-defined. Since L n = for X n = 0, L n is well-defined 
provided that L e n exists for X n > 0. Note that ]im v f y Jt?{v + n y mt (y — u) , v) = > ^- ln ^ for y > 0. 
This fact together with Lemma [28] imply the existence of L l n for X n > 0. So, L l n is well-defined. From the 
definition of L n , it can be seen that 

{A < L e n , ~X n = 0} = { A < X n , Jtv (x + -^(I„ - A), a) < — In %, X n =o} = 0, 
L \ n V mi ) mi 2 J 

{A<4, X„>0}c(a<X„, ^ p fA+— ^(X„-A),a) <— ln^, X„>ol. 

[ I r»V mi J mi 2 J 

This implies that {A < L l n ) C {A < X n , „# P (A + ^(X„ - A), A) < ^ lnf }. 

Next, consider Pr{Lfj < A for all n £ JV}. Since lim^oo t(ln j — 1) = — oo, there must exist an e* > 
such that (A + e*, A) = ^ ln ^. Note that ^#p(A + e, A) is decreasing with respect to e > 0. Therefore, 
from the definitions of L l n and e* , we have that {A < L l n } C {A < X n , ^#p(A + n ™ (X n — A), A) < 
^ In C {A < X n , ^(X„ - A) > e*} C {I n > A+ ( " v 7 )gt }. This implies that {i£ < A} D |A„ < 
A + ( " v 7 )£ * } for all n 6 ./K. Hence, {L e n < A for all n £ ,yV} D {X n < X + ( " v ^^ £ * for all n £ ./K}. 
It follows from Lemma [3D] that Pr-ji^ < A for all n G Jf} > Pr{A„ < A + (" v ™') £ * for all ne/}> 
1 — exp (m^^p(A + e*, A)) = 1 — f for £ £ N. This completes the proof of the lemma. 

□ 

Lemma 33 For n £ ,Jf and £ £ N, define U e n = sup{v G (X n ,oo) : ^# P (iy- ^^ r (i/-X n ), i/) > ^hi^f}. 
Then, Pr{U* > A for alln G ^} > 1 - % for I G N. 

Proof. First, we need to show that f/„ is well-defined. Note that lim^y ^i~p(y — n y mt (y — y) , v) = > 
^ ln y for y £ [0, oo). This fact together with Lemma [28] imply the existence of U^. So, U* is well-defined. 
From the definition of U*, it can be seen that {A > U*} C {A > X„, ^ P (A- ^_(A-X„), A) < ^ ln f-}. 

Next, consider Pr{[/„ > A for all n £ jY} for two cases as follows. 

Case A: exp(— mg A) < If- 

Case B: exp(— m^A) > 4f . 

In Case A, there must exist an e* £ (0, A] such that „# P (A — £*, A) = ^- ln 4f . Note that ^ P (A - e, A) 
is decreasing with respect to e G (0, A). Therefore, from the definitions of [7^ and £*, we have that 
{A_ > ^} C {A > A„, ^p(A- ^j(A- X„),A) < 4ln|} C {A > X„, _^_(A-X n ) > e*} C 
{X n < A - (" v ^) g " }, This implies that {[/^ > A} D {X n > A - ( " v ^ g)e * } for all n £ jV . Hence, 
{Ui > A for all n £ ,yV} D {X n > A - ( " v " f)e * for all n £ jT}. It follows from Lemma [3T] that 

Pr{t/J5 > A for all n G Jf} > Pr{A„ > A - ( " v "; f)e * for all n £ ,yY) > 1 — exp (m^ P (A - £*, A)) = 1 - % 
for £ G N. 

In Case B, we have {A > X n , J( V [X - ^(A - X n ),X) < ^lnf } = {A > X„, -A < ^ P (A - 

^jj(A - A„),A) < ^ln^} = 0. It follows that {A > U l n } = for all n £ JV . Therefore, Pr{J7^ > 

A for all n £ ^V} > 1 - E„ e .^ Pr { A > t 7 ™} = 1, which implies that Pr{[^ > A for all n £ J/} = 1 for 
£ £ N. This completes the proof of the lemma. 

□ 

Lemma 34 

{X n - e < L e n < Ui < X n + e} 

= f^P (X n +s- -^,X n +£ )<±ln S 4,,#Jx n - e + ~^,X n - £ )<iln|) 
[ \ n V / mf 2 \ n V m« / m< 2 J 

/or aline Jif and £ £ N. 
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Proof. From the definitions of and L n , it is clear that 

X n - e + —^,X n - e J < — In ^, X n - e < } = (X n -e<0,X n -e< L L n } 
for n G jY '. By Lemma |2"51 and the definition of L l n , 

X n -e+ -^,X n - s) < — In ^, X n - e > \ = {0 < X n - e < L l n } 
for n G jV. It follows from (|33j and (f34l) that 



n V 77i£ / 2 1 



for n G ^/K. 

By Lemma [2H and the definition of {/„ 



{X„ + e > t/*} = ^ P ( X n + e - -^,X n + e] <—ln^} (36) 

n V / me 2 1 



for n G „/f . Finally, combining ([35]) and (f55|) proves the lemma. 

Making use of Lemmas [34l and l29l we have the following result. 
Lemma 35 

(X n - e < Li < U l n < X n +e} = \Mv (X n +e rr~ + e ) < — In &l 



□ 



n V me J mi 1 

for all n e ,yY and I G N. 

By a similar argument as that for proving Lemma [f"5l we have established the following result. 

Lemma 36 Define C n = sup £gN L n and U n = inf^ g N for n G jV . Then, Pr{£„ < fj, < U n for all n G 
Jf} > 1 — 5. 

Lemma 37 Pr{X„ — £ < £ n < W n < -X"n + £ /or some n G =yf } = 1. 

Proof. By the definition of C n and U n , it is sufficient to show that Yr{X mt — e < L~j < f/~ < + 
£ for some t! £ N} = 1. In view of Lemma [35l this is equivalent to show that Pr{^#p (X mei X me + e) < 
;Mn4f for some £ G N} = 1. Note that Pr{^# P (X me , X mj! + e) < ^ ln 4f for some ^ G N} > 
lim^oo Pr{./#p (JT m< ,,X mf +£) < ^ln^}- To complete the proof of the lemma, it remains to show 
that lim^oo Pr{^#p (X mi , X me + s) < ^- ln ^} = 1, which is accomplished as follows. 

Let < r] < 1. Noting that ln ^ — >■ > J{-p(^, A + e) as ^ — > oo, we have that there exists an 
integer k such that ./#p(-, - + e) < — In 4f for all £ > k. For ^ no less than such k, we claim that 
z > — if ./i#p(z, z + e) > ln ^ and z G [0, oo). To prove this claim, suppose, to get a contradiction, that 
z < —. Then, since .-# P (z, z + e) is monotonically increasing with respect to z > 0, we have .4^p(z, z + e) < 
J(y{-, - +e) < — ln4f, which is a contradiction. Therefore, we have shown the claim and it follows that 
{^ P (X me ,X me + e ) > -i-ln^} C (X me > A} for £ > K . So, Pr{^ P (X mi ,X me + e) > ^ln^} < Pr{X m , > 
A} < exp(-cmf), where c = — ./#p(A,A) and the last inequality is due to Chernoff bounds [3j. Since 
■mt — > co as £ —> oo, we have lim^^oo Pr{^p(X mf , X lri£ + e) < ^ ln 4f } = 1. This proves the lemma. 

□ 
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Now we are in a position to prove that stopping rule E ensures the desired level of coverage probability. 
From Lemma 1351 we know that the stopping rule implies that "continue sampling until {X n — e < L n < 

< X n + e} for some £ £ N and n £ Jf" . We claim that this stopping rule implies that "continue 
sampling until {X n — e < C n < U n < X n + e} for some n £ jV" . To show this claim, we need to show 

|J |J {X n -e<Li<Ui<X n + e}C [j {X n - e < C n <U n <X n + e} , 

which follows from the fact that \J em {X n — e < L e n < Ui < X n + e} C {X n — e < C n < U„ < X n + e} for 
every n £ ,JV . From Lemma 1371 we know that the sampling process will eventually terminate. It follows 
from Lemma l36l and Theorem [1] that Pr{|X n — A| < e} > 1 — S. 

A. 6 Derivation of Stopping Rule F 

We need some preliminary results. 

Lemma 38 Jg? j^) > J(? [ {1 ~l+ r e e)y , j^) f or £ e (°> 1). r e (°> l \ and V > °- 



Proof. For simplicity of notations, let 8 
d^ P (z,8) a^ P (w,d) 



y ,q _ y 

l+e> 



de 

y 



de 

2 {l-r)\n 7 - 



+ 



(1+e) 2 8 ' (1-e) 



, z = T7 — I - tt^ an d w 

1— e' 1+e 1+e 



V (l-r)ln W 



_Ji_ _ T3L, Note that 



y w — ?? 



■d (1-e) 2 ■d 



where h(t) = (r - t) 2 ln(l + 1) - (r - <) 2 . Using ln(l + i)=i-V + T~T 



Kt) = {r-tf + £ 



and 



> 2t 2 



4r 2 
1 - r 

" 2r 2 



r 2 U 2 



2r 



1 -r 



2fc 2fc - 1 2(& - 1) 
t 2 



> 2t 2 



2r 2 



fe=2 

r 2 2r 1 

y + y + 2; i-t 2 

r 2 2r l\ (re) 2 
T + T + 2/ 1 - (re) 2 



1 



for Itl < 1. we have 



±2k 



> 



l-r 2 

r 2 i 2 [18 + 6r - (2r + 6)e 2 - 13(re) 2 - 3r 3 e 2 ] 

6(1 -r)[l - (re) 2 } 
r- 2 t 2 [18 + 6r - (2r + 6) - 13r - 3r] _ 2r 2 t 2 
6(1 - r)[l - (re) 2 ) ~ 1 - (ref 



> 0. 



This shows d-^p(>,9) _ g^^W > q ^he proof of the lemma can be completed by making use of this 



result and the observation that ^p(z,8) — ^p(w,'d) for e = 0. 



□ 



Lemma 39 

{^<Li<ui<^}2{x n >0,^(^(l + ^),^)<±lJA 
[1 + e 1 — e J ^ \ 1 + £ \ n\/mejl + £j me 2 J 

for all n £ JV and £ £ N. 
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Proof. From the definitions of and L n , it is clear that 

( ^ ( 1 + , < - ^ f , Xn = 0} = (X„ = 0,^<Li) (37) 

for n £ .y^. By Lemma [28] and the definition of L„, 

^ + -J^-l > ^r-) < — In I, X, > ol = {0 < X„ < Li} (38) 



1 + e \ nV mg J 1 + e J mi 2 
for n 6 +K. It follows from (1571) and (Ell) that 



X " <£lj = W^fl + ^V^U^ln^i (39) 



n 



l + e n J \ \ 1 + s \ nV mi ) ' 1 + e / 2 
for all n € From the definitions of and U„, it is clear that 

^>0,^ P (^(l-^),-^)<J-ln|)c{^> C /A (40) 
\ 1 — e \ n\l m,g ) \ — e J mg 2 J ^ 1 — e ) 

for n 6 o/K. Finally, combing (|59|) . (|40]) and using Lemma 1551 complete the proof of the lemma. 
Lemma 40 Pr{f^+. < C n < U n < ^ /or some n € ^} = 1. 

Proof. By the definition of C n and U n , it suffices to show that Pr{^^ < L e mf < U^ le < f° r some i G 
N} = 1. From Lemma 1591 it can be seen that 

Pr {rf-e- £ ™« - u ™< - r=7 for somc 1 e N 

> Pr \x mt > 0, Jtv (x me , < — ln % for somc i G N 

This inequality and Bonferroni's inequality imply that 

Pr I < ^ < ^ < *2£ for somc £ e N 

> lim Pr{X„ lf > 0} + lim Pr [X mt , < — In § 1 - 1. 

Since A > 0, it follows from the law of large numbers that lim^oo Pr{JT m{ > 0} = 1. To complete the proof 
of the lemma, it remains to show that lim^oo Pr |^#p ^X mtl -j-^rj < In y| = 1. This is accomplished 
as follows. 

Let < r/ < 1. Noting that ^- ln 4f is negative for any £ > and that ^ ln > JKvljiX, ^) as 
£ +► 00, we have that there exists an integer k such that Jt-p{r\\ < ln 4f for all £ > k. For no less 
than such k, we claim that z < rj\ if _#p(z, y^) > ln 4f and z G [0, 00). To prove this claim, suppose, 
to get a contradiction, that z > rj\. Then, since ^p(z,j^) is monotonically decreasing with respect to 
z e (0,oo), we have ^#p(z, j^) < < ^ln^-, which is a contradiction. Therefore, we have 

shown the claim and it follows that {JKv{X mi , -j^f) > ;^7 m if} Q {X me < ??A} for i > k. So, 

Pr (uT P ( X me , Zzl) > — ln *±\ < Pr(X me < V \} 
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for large enough £. By the law of large numbers, Pi{X me < r/X} —> for sufficiently large I. Thus 
lim^oo Pr |./#p (^X me , -f^f^j < ^ hi % j = 1. This proves the lemma. 



□ 



Now we are in a position to prove that stopping rule F ensures the desired level of coverage probability. 
From Lemma [39l we know that the stopping rule implies that "continue sampling until {frpj < L^ n < < 
yz^} f° r some £ <E N and n £ jY" . We claim that this stopping rule implies that "continue sampling until 



{jzE; < £ n < Un < T^} f° r some n S jV" . To show this claim, we need to show 



U U 



<L£<#<4=-> 



n — n 



l-e 



l-e 



which follows from the fact that U £SN < < < ^} ^ < £ n < W„ < y^j for every n G ^K. 
From Lemma l40l we know that the sampling process will eventually terminate. It follows from Lemma 136 
and Theorem □ that Pr{|X n - A| < s\} > 1 - S. 



B Proof of Theorem [2] 

To show (flj, note that 

= 2(, - ,)[(! - ,) 2 + 6] + (1 - Z )[0 - (1 - ,) 2 ] In (l + {z -^ 1 + _%~' i)2] ) 



[(i-^ + g] 2 ^'^ 



Oft 
:n : 

[(l-tif + ef d*i>(z,»,9) 

(1 - z)[9 - (1 - m) 2 ] ^ 



If 6> < (1 - ii) 2 , then < 0. If 6* > (1 - /i) 2 , then 



= In 1 



(z - + (1 - m) 2 ] \ 2( 2 - - m) 2 + 0} 



9(1 -z) J (l-z)[0-(l- M ) 2 ] 

(2- M )[0 + (i-^) 2 ] 2(z~ f i){(i~ f if + e] 



(l-z) 



(i - z)[e - (i - ^} 



(z-»)[6 + (l-tf] 
9(1 -z) 



for < // < z. 

To show note that 

djA^ff) = (l- M )(l-z) 
96> [(1 - /x) 2 + 6>] 2 

To show note that 



In 1 



(z-fi)[(l-fi) 2 +ff\\ (z - M )[(l -/i) 2 + i 



0(1 -z) 



/ 2 . a ^ d<p(z,n ,6) 



= 2(/x - z)(/i + 9) + z(fi 2 - 6) In 1 + 



0(1 - z) 

( M -z)(e + / i 2 r 

<9z 



If 6> < /i 2 , then > o. If 6 > /i 2 , then 



(/x 2 + 0) 2 dip(z, n, 
z(9-/i 2 ) dfi 



for < z < /x. 

To show (gj, note that 

d<p(z,(i,\ 



2(n - z)(/j, 2 +i 
z(fl-A* a ) 



In H 



( M -2)(fl + /i 2 ) 
<9z 



2(/x - 2 )(/i 2 +61) Qx~ g)jg + )i 2 } (/x- gKg + m 2 ) 

2(6»-/i 2 ) 02 - 02 



z/x 



(9(9 



(/x 2 + 



In 1 



(/i-z)(/x 2 + 6')\ ( At -z)( A i 2 + 



< 



for < z < (j,. This completes the proof of the theorem. 



< 0. 
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C Proof of Theorem [3] 

Define L(X, 9) = inf iv g [0, X] : ^(X, i/, 0) < ^-j and U{X, 9) = sup h>e\X,l\: <p(X, v,6)<^\ for 

< < For simplicity of notations, let Gy(z) = Pr{X > z}. From the definition of L(X, 9), HoefTding's 
inequality, and (I]) of Theorem [21 we have 

f ln^ 

{L(X, 9) > M } C U < X, i>(X, fj,, 9) > 

H<X, V(X,/z,0) > G X (X) < exp(-nV(X,/i,0))} C |g x (X) < | 

It follows that Pr{L(X,6>) > fi} < Pr {G X (X) <§}<§. In a similar manner, we can show that 
Pt{U(X,9) < fi} < |. Therefore, by Bonferroni's inequality, 

Pr{L(X, 0) < /i < U(X, 0)} > 1 - y . (41) 

Now define U(X,V,fx) = sup |i9 e [W„, |] : <A(W M ,i?) < ^1} for < /i < 1. Let F = (X - n) 2 . Then, 
y is a random variable with mean 9 — a 2 . Define Y = 'Su—ii^—tl jt can be checked that Y = W„. 

n P 

For simplicity of notations, let Fy(y) = Pr{F < y}. By HoefTding's inequality, we have Pr{y < y} < 
exp (—n<fr(y , 9)) . From the definition of U(X,V, /i), Hocffding's inequality, and the fact that <^(W M ,0) is 
non-decreasing with respect to 9 > W^, we have 

f ln^ 

{U(X,V, f j,)<9}clw„<6, 4>(W„,9)>-^- 

W,. < 0, 4>{W^9) > —L, Fy(Y) < cxp(-n0(W M ,0)) j> C <{ F F (y) < 



It follows that 

(51 J 



Pr{W(X, V, /i) < 9} < Pr |%(y) < - 1 < - . (42) 

Define random region 

V(X,V) = {(i/,0) : < v < 1, < < - v), L(X,-d) < v < U(X,ti), U{X,V,v) > j?} . 

Then, from Bonferroni's inequality, Pr{(/x,0) G £>(X,F)} > 1 - 6. By © of TheoremEJ and the fact 
that 4>(W„, $) is non-decreasing with respect to $ > Wi,, we have £>(X, V) = AU B, where 

A= v e (0,T), g (0,1/(1-1/)], max^(X,t/,tf), 0(W„,0) I^w,,}] < ^} , 

f In 5 

B= Ui/,!?) :i/G (X,l), t? g (0, i/(l - i/)] , max [v?(X, i/,i?), <f>(W„,d) I { # >Wv} ] < 

Finally, the theorem follows from the observation that V) = ^4 U B. 

D Proof of Theorem [4] 

Define 

f In ^ "1 f In ^ "1 

L(X,6)=mfiu G [0,X] : ip(X,v,9) < L £/(X,0) = sup j 1/ G [X, 1] : <p(X,v,9) < \ 
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for < 9 < \. Define 



In I 



C(X,V,n)=va£\&e [0,W P ] :<p(W„,§) < — }, U{X, V, n) = sup id € 
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n 



for < \i < 1. By a similar method as that for proving (|4Tj) , we can show that Pr{L(A, 9) < fi < 
U(X, 9)} > 1 — |. By a similar method as that for proving (l4"2"j) . we can show that 

Pr{£(X,V,n) > 6} < i, Pt{U(X,V,v) <&}<\- 

Therefore, by Bonfcrroni's inequality, Pr{£(X, V, /i) < 9 < U(X, V, /Lt)} > 1 - f . Again by Bonfer rom s 
inequality, 

Pr{L(X,9) < n < U(X,6), C(X,V,fi) < 9 < U(X,V, n)} >1-S. 
By ([TJ, ([3]) of Theorem [5] and the unimodal property of —4>(Wfj,,8) with respect to 8, we have that 

&(X,V) = : <i/ < 1, <7? <i/(l- v), < v < U(X,ti), £{X,V,v) < ■& < U(X, V, v)} , 

which implies that Pr{(/x, a 2 ) £ @(X, V)} > 1 — S. This completes the proof of the theorem. 
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